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I/Q MIMO Detection for Single Antenna Interference Cancellation 
TECHNICAL FIELD 

[0001] The field of the invention is that of processing interfering RF signals in a 
system having two or more signals that interfere with one another, in 
particular interference cancellation. 

[0002] The present invention applies to Enhanced Data Rates for a GSM 

Evolution (EDGE) system in which two different modulation formats are 
used namely binary Gaussian Minimum Shift Keying (GMSK) and 8 level 
Phase Shift Keying (8PSK). 

BACKGROUND OF THE INVENTION 

[0003] As more signals are sent within a fixed amount of RF spectrum, there 
are more occasions when signals interfere with another signal that is 
intentionally transmitted along the same channel, rather than being 
random noise. When frequency spectrum is available on a limited basis 
as in cellular or PCS systems, the same frequency is reused in 
geographically distinct regions (or cells) to maximize the spectrum 
efficiency. The overall system capacity is generally limited by the 
interference caused by the reused frequencies. 

[0004] Interference cancellation (IC) is traditionally achieved by the use of 

multiple antennas at the receiver terminal. However addition of antennas 
at the receiver significantly adds to the receiver cost. On the other hand 
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IC using a single antenna is an attractive way to increase system 
capacity since it eliminates the need for additional RF hardware cost 
since the algorithm can be implemented digitally in base band. 



[0005] The recently discovered single antenna interference cancellation (SAIC) 
algorithms are specifically tailored for GMSK signal detection in the 
presence of GMSK interferes. The ICUPC 1 998 paper "Cochannel 
Interference Cancellation within current GSM Standard" written by Trigui 
and Slock exploits the special properties of GMSK modulation for SAIC. 

[0006] A number of other SAIC approaches have also been proposed in the 

literature. Examples include: Ottersen, Kristensson, Astely, "A receiver", 
International Publication Number WO 01/93439; Arslan, Khayraliah, 
"Method and Apparatus for Canceling Co-Channel Interference in a 
Receiving System Using Spatio-Temporal Whitening" International 
Publication Number WO 03/030478 A1; Meyer, Schober, Gerstacker, 
"Method for Interference Suppression for TDMA-and/or FDMA 
Transmission", filed December 19, 2001. Also of interest are B. 
Picinbono and P. Chevalier, "Widely Linear Estimation with Complex 
Data," IEEE Trans. On. Signal Proc, vol. 43, pp. 2030-2033, Aug, 1995; 
W. H. Gerstacker et al, "Equalization with Widely Linear Filtering," 
ISIT2001; W.A. Gardner, S.V. Schell, "GMSK Signal Processors For 
Improved Communications Capacity and Quality, U.S. Patent No: 
5,848,105, Dec, 8 1998; 

[0007] The above-mentioned prior art is relevant for SAIC for GMSK signals or 
any other real modulation signals. However the literature has not dealt 
with IC techniques involving both real and complex modulation 
alphabets such as GMSK and 8PSK signals, which are encountered in 
GSM/EDGE systems. If SAIC gains are to be maximized in EDGE 

-2- 



deployments it is necessary to develop a mobile receiver that has 
interference cancellation capability irrespective of the interference 
modulation type. 

[0008] IC methods are generally categorized into two classes namely blind IC, 
and Joint detection (JD). By "blind", we mean that the channel 
parameters of the interfering signal are not known at the receiver and 
that the interfering signal is not detected together with the desired signal 
as in JD. The paper titled "The Finite-Length Multi-Input Multi-Output 
MMSE-DFE written by Al-Dhahir and Sayed may be a useful approach 
for JD, though it does not handle GMSK-8PSK IC. Significant 
improvement would be required to extend this technique to the 
GMSK-8PSK IC problem. So far no prior art is reported on IC involving 
8PSK and GMSK signals. 

[0009] The art would benefit from a system and method that is able to suppress 
interference between 8PSK and GMSK signals. 

SUMMARY OF THE INVENTION 

[001 0] The invention relates to a method for using interference cancellation 
with various combinations of signals; e.g. GMSK and 8PSK in various 
combinations. 

[0011] A feature of the invention is the detection of real and imaginary parts of 
an incoming set of signals that include at least one complex signal. 

[0012] A feature of the invention is the rotation of the incoming complex signal, 
if necessary (as in the case of GMSK) using parameters supplied by 
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previous estimations or by a base station so that the imaginary 
component of the GMSK signal is eliminated. 

[0013] Another feature of the invention is using an MIMO process on at least 
three input signals: the real GMSK signal and the real and imaginary 
components of the 8PSK signal. 

[0014] Another feature of the invention is a high data rate system in which 

deliberate interference is created by transmitting multiple signals from 
the same base station through spatially separated antennas. At the 
receiver, signal separation is performed through interference 
cancellation, without having multiple receive antennas. 

[0015] For instance, the peak data rates for GPRS can be improved by a factor 
of 2 by transmitting two GMSK signals, and EDGE peak data rates can 
be improved by 4/3 or 2 by transmitting GMSK-8PSK or 8PSK-8PSK 
signals simultaneously. 



BRIEF DESCRIPTION OF THE DRAWINGS 

[0016] Figure 1 illustrates a block diagram of a system employing the invention. 

DETAILED DESCRIPTION 

[0017] Figure 1 illustrates a simplified block diagram of a portion of a 

telecommunications system in which base station 10 communicates with 
three or more mobile stations 100, 200 and 300. Station 10 is 
connected to a network 20, which may be a switched telephone (voice 
or data) network having as input voice handsets 22 and/or data 
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terminals 24. 

[0018] Mobile stations 100 - 300 have single or multiple antennas 1 12, 212 and 
312, respectively and communicate with the base station over channels 
110, 210 and 310. Illustratively, base station 10 has two or more 
antennas 12 and 14. Base station 10 is capable of transmitting GMSK 
and/or 8PSK signals. Within base station 10, RF unit 16 handles the RF 
(and IF) signals, while processor unit 18 handles manipulation of the 
baseband signals. Counterpart units 116, 216 and 316 for RF and 118, 
218 and 318 for baseband are located within the mobile stations 100 - 
300. 

[0019] The invention exploits the special properties of GMSK/BPSK signaling 
for improved signal detection when GMSK/BPSK and 8PSK signals 
interfere with each other. Unlike conventional detection methods, the 
invention relies on novel detection methods that detect the real (I) and 
imaginary (Q) parts of a combination of complex and real modulation 
alphabets for improved signal processing. 

[0020] We adopt the following notation throughout. The desired modulation 
type is mentioned first, followed by the dominant interferer modulation 
type e.g in station 100, the signals are GMSK - 8PSK; in station 200, the 
signals are 8PSK - GMSK; and in station 300 the signals are 8PSK- 
8PSK. The receiver processing assumes a linearized GMSK signal 
model as specified in the papers by Laurent , "Exact and Approximate 
Construction of Digital Phase Modulations by superposition of Amplitude 
Modulated Pulses (AMP) and Kaleh, "Simple Coherent Receivers for 
Partial Response Continuous Phase Modulation". 

[0021] In an illustrative example, station 10 is transmitting to mobile unit 100 
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and to mobile unit 200 using 8PSK and/or GMSK on the same 
frequency band for both transmissions, so that each mobile unit will 
have interference. The type and significance of the interference will 
vary. 

[0022] In the following discussion, the channel parameters h are assumed to be 
known. When the interfering signal comes from the same base station 
as the data, the base station can transmit to each mobile unit the 
relevant parameters of the interfering channel, thereby reducing the time 
spent by the mobile unit in estimating those parameters. When the 
interfering signal comes from a different base station, the mobile unit will 
have to perform a conventional estimation process. 

[0023] In a first example, the base station transmits data on the same channel 
in both GMSK and 8PSK formats to mobile station 100. Station 100 
uses techniques according to the invention to detect the data in both 
formats, thus increasing the bandwidth. 

[0024] In a second example, the base station transmits two different signals 

(GMSK or 8PSK) using two antennas to two mobile units and cochannel 
interference (8PSK or GMSK) comes from the same base station. 

[0025] In a third example, the base station transmits a signal (GMSK or 8PSK) 
to a mobile unit and (8PSK or GMSK) cochannel interference comes 
from a different base station. 

[0026] After converting from RF to baseband, a linearized GMSK signal 

representation for simplified receiver processing is used to de-rotate the 
received signal, which is a sum of one GMSK and one 8PSK signal, with 
a factor exp (-jk7i/2 ) (where k is the symbol time index), such that all the 
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component GMSK signals will be forced to be binary modulated. There 
are three data streams - a real-only stream from the GMSK data and a 
real and imaginary data streams from the 8PSK data. 

[0027] This disclosure is directed at single antenna reception and joint 

detection of at least two signals having same or different modulation 
size for e.g., GMSK-GMSK, GMSK-8PSK, 8PSK-GMSK and 8PSK- 
8PSK scenarios or detection of a single 8PSK signal in the l/Q domain 
with additional l/Q whitening capability. Although we are primarily 
interested in a single receiver antenna, we present the algorithm for the 
general case of N receiver antennas. Further we assume that the 
received signal comprises M, GMSK co-channel signals and M 2 8PSK 
signals and an additive noise component, which may include the sum of 
residual co-channel or adjacent channel interfering signals and thermal 
noise. Then the complex base band signal at the kth sampling instant 
can be represented as: 

[0028] a^ZE^/^+ZieV,,/^ +"Z, q = l,,NS 

,1=1 m =0 ,2=1 m=0 

(D 

7rJc 3ttAt 

[0029] where <f> k = — , 0 k = are the modulation dependent constellation 

2 8 

rotation factors as defined in the GSM/EDGE standard. For each symbol 
index k , the receiver collects NS samples corresponding to an over- 
sampling factor of s at in case of N receiver antennas. Further 
/z^and denote the channel coefficients for m th tap, at the q\h 
over sampled observation, corresponding to /, th binary symbol source 
a k- m j { . and ' 2 8PSK complex symbol source b k _ mh respectively. Further 
we assume that the channel length is equal to L + 1 for all signals. 
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[0030] When the data are contained in one format, then one sum in the 

preceding equation is the data and the other sum is the interfering signal 
to be subtracted. According to the embodiment illustrated, when the 
data are contained in two formats (GMSK and 8PSK), the interfering 
signals are in the r\\ ; i.e. this example is not directed at the case when 
the data are GMSK and 8PSK and there is significant interference. 

[0031] Those skilled in the art will be able to generalize the example presented 
to handle interference by adding sums to equation 1 with channel 
coefficients that represent the interfering channel. 

[0032] The base band received signal, which is a sum of at least one GMSK or 
at least one 8PSK signal, is first de-rotated with a factor e~ M such that 
all the component GMSK signals are forced to be binary modulated. The 
de rotated base band signal can be represented as 

Mi L M 2 L _ 

[0033] e~* r Kq = £ £ h^a^ + £ £ W + "** 

/1=1 i«=0 /2=1 m=0 

(2) 

[0034] where the de-rotated quantities are denoted by the tilde symbol. Since 
8PSK is a complex modulation alphabet, multiplication with a complex 
de-rotation factor yields a constellation rotated 8PSK modulation. 

[0035] After de-rotation, we split the real (I) and imaginary (Q) parts of the de- 
rotated GMSK-8PSK composite signal and represent in column vector 
format as: 

M\ L M 2 L 

[0036] y kM = £ E n ( X\>-. M e Jh ~ + Z Z ^\^ J ^ + *I 

il=l m=0 /2=I m=0 

(3) 
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[0037] where y kq = 

column format, H','l = 



Re 
Im 



is the l-Q received samples in stacked up in 



. J/ 



is the l-Q parts of GMSK channel 



stacked up in column format, c k _ mJj 



is the column vector 



filled with l-Q parts of 8PSK symbol, and = 



is 



the corresponding 8PSK l-Q matrix channel, and h q k = 
residual noise vector after l-Q split. 



'Re 









is the 



[0038] The 8PSK l-Q matrix channel is orthogonal due to the inherent l-Q 
orthogonality. 

[0039] Note that after de-rotation and l-Q split, we have three data modulation 
formats -binary data streams from GMSK sources and real and 
imaginary data streams from the 8PSK sources. 

[0040] Further y k is represented in compact matrix form as: 



[0041] y M = £Hl*^+*Z 



(4) 



m=0 



[0042] in which we further stacked the l-Q parts of GMSK and 8PSK channel 
and data matrices in matrix format as: 

[0043] H'-flli* h£ .. H£*] and 

f is a column vector filled with l-Q 
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parts of GMSK and 8PSK symbols. Note that the superscript symbol T 
denotes a matrix transpose operation. 



[0044] We further stack the over samples corresponding to £th sample from all 
receiver antennas in column format as: 

[0045] y A =[y- ... yj, ... y^f 

[0046] Then we get the following compact signal definition 

[0047] y k =£H m d,_ m +n t (5) 

m=0 





m 






i. where H m = 




and n k = 






m 







We then filter the received vector with a set of feed forward weights w 

such that the filtered signal has an impulse response with most of the 
energy concentrated in the first few taps. This type of filter is also referred 
to as a channel-shortening filter in the literature. The weights are chosen 
to minimize the error term defined as = w T y k+Lf[k - Bd kk _ L 

[0048] where B denotes the l-Q MIMO feedback filter. 

[0049] We use the notation where y k+lf . k is a vector that contains the blocks 
yA+^v-y* » where each block is a vector of size 2NS x 1 . Note that we 
have assumed a finite length feed forward filter of length L f . The feed 
forward filtered received signal x fc = w T y k+L/:fc has the following signal 

L 

structure = ^ B / d *-/ + e * 

/=0 

[0050] We further define the error auto correlation matrix as R ee = E[e*e[] 
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[0051] We then optimize the filter coefficients according to the minimum mean 
square error (MMSE) criterion by minimizing the trace of the error 
autocorrelation matrix R ee 

[0052] w opt ,B opt = arg min Trace(R ee ) such that first tap of the feedback filter 

w,B 

matrix is constrained to be B 0 = I or a lower triangular matrix. 

[0053] The pre filtering step thus allows a decision feedback equalizer (DFE) or 
a reduced state sequence estimation (RSSE) algorithm can be applied. 

[0054] A subsequent reduced state soft output sequence estimation algorithm 
(using one of MAP/Max-Log-MAP/SOVA algorithms) jointly detects the I 
/Q symbol streams using the shortened impulse response B opt . 
Furthermore, the constellation rotated 8PSK symbol set will be used in 
the equalizer branch metric computations. The branch metric is given by 



[0055] d\k) = 




[0056] where d indicates the state dependent path history vector in a joint 
RSSE algorithm. We note the difference between the novel branch 
metric above made up with l-Q parts of the composite signals, and a 
conventional branch metric defined using complex quantities. 

[0057] The result of the foregoing is that the desired signal emerges from the 
MIMO process, with the interfering signal removed as part of the 
sequence estimation process. A similar procedure is followed in mobile 
stations 100 and 200. 

[0058] The l/Q MIMO MMSE framework described above includes the following 
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scenarios as special cases. 



[0059] By setting M1 = 0 and M2 = 1 , the invention can be configured as an 
8PSK blind l/Q interference suppression receiver when a GMSK 
interferer is present. 

[0060] The invention can be configured as GMSK-8PSK or 8PSK-GMSK l/Q 
MIMO MMSE joint detection receiver if we let M1 = 1 and M2 = 1 

[0061] The invention can be configured as 8PSK-8PSK l/Q MIMO MMSE 

receiver if we let M1 = 0 and M2 = 2. In this mode the receiver jointly 
detects two 8PSK signals and rejects any residual GMSK interference 
through l-Q whitening. 

[0062] The invention can be configured as GMSK-GMSK l/Q MIMO MMSE 
receiver if we let M1 = 2 and M2 = 0. In this mode the receiver jointly 
detects two GMSK signals and rejects any residual GMSK interference 
through l-Q whitening. 

[0063] Many other possibilities include for instance joint detection of 3 or more 
signals (for example 2 GMSK and one 8PSK) in l-Q domain. 

[0064] In a practical implementation, the receiver can be made to switch 

between the above realizations by examining the desired and interfering 
signal channel estimates. Those skilled in the art will readily be able to 
set up criteria for switching, based on their needs. 

[0065] Joint Channel Estimation 

[0066] In an optional version of a system according to the invention, a receiver 
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sequentially estimates the desired and dominant interfering 
transmissions and applies the cancellation process described above. 

[0067] In this section we outline a joint channel estimation method that 

estimates the desired and dominant interferer signal channel impulse 
responses. The channel estimation method blindly (without a priori 
knowledge) identifies the dominant interferer modulation type (GMSK or 
8PSK) and its training sequence (TRS) number during the estimation 
process. The modulation identification is done by exploiting the e jTck/2 , 
e j37ik/8 cons te||ation rotations associated with GMSK and 8PSK 
modulations respectively, where a TRS identification is made possible 
by formulating a maximal likelihood metric (or a simple correlation or 
least squares or LMMSE metric) over all possible desired and 
interference training sequence pairs. 

[0068] In one embodiment of the invention, the receiver may first identify the 
desired signal modulation type and training sequence number by 
searching through the known relatively small number of training 
sequences. Next, the receiver analyzes the residual signals to identify 
the type of interference that is dominant. 

[0069] In more detail, the estimation method consists of the following steps: 

[0070] First, de-rotate the received signal by e iM2 as above. The de-rotation 
process forces the GMSK component signals to be binary. 

[0071] Then solve for the channel estimate using one of maximal likelihood 
(ML), or least minimum mean square estimation (LMMSE) or another 
convenient method of optimization for all possible desired and 
interference training pairs. Solve for the channel estimate H, using the 
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following ML metric 



[0072] m i n 



y*-ZH,d, 



k-m 

m=0 



(7) 



[0073] In another embodiment of the invention, a reduced complexity 

implementation may be devised such that the interfering modulation 
type and TRS number are estimated sequentially, then a final ML joint 
channel estimate can be made once all modulation types and TRS 
numbers are estimated. 

[0074] Implementation Alternatives 

[0075] The invention provides many implementation alternatives: 

[0076] The preceding discussion has implicitly assumed that the interfering 
signal is unwanted. In one embodiment of the invention, the base 
station transmits two signals on the same channel, with both the signals 
being part of the message. In that case, both signals are important and 
need to be detected accurately. 

[0077] The base station transmits the signals on two (or more) different 

spatially separated antennas. The receiving station has only a single 
antenna and applies one of the techniques described above to separate 
the two signal streams and discard interference between the streams. 

[0078] When both the signals are GMSK, the data rate can be doubled. When 
one signal is GMSK and the other is 8PSK, the data rate can be 
improved by a factor of 4/3. 

[0079] If 8PSK is the desired signal and GMSK is the source of interference, 
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the invention can be configured as an 8PSK blind l/Q whitener that 
whitens the GMSK signal as colored noise in l/Q space. 

[0080] Alternatively, if 8PSK is the desired signal and GMSK is the source of 
interference, the invention can jointly detect desired and interfering 
signals in l/Q space. Similarly, if GMSK is the desired signal and 8PSK 
is treated as CCI, the same algorithm can be used for interference 
cancellation (IC). If 8PSK is the desired signal and the CCI comprises a 
dominant 8PSK and a residual GMSK interferes the algorithm can jointly 
detect two 8PSK signals and also whitens the GMSK interference in l/Q 
space 

[0081] For example, an alternate version of the invention includes a subsystem 
for detecting whether the transmission-receiver system is in a first 
transmission mode in which the interfering signal is to be discarded (i.e. 
the second signal is directed at another mobile station, etc.) or is in a 
second transmission mode in which the first signal and the second 
signal are both to be processed as data (i.e. the high data rate modes 
discussed above); and 

[0082] processing the second signal accordingly (e.g. discarding or saving it). 

[0083] A transmission system may be set up with a convention such that the 
default primary format is GSM and a secondary format of e.g. 8PSK 
indicates that the high data rate mode is on, so that the second signal is 
to be saved and combined with the first signal; while a secondary signal 
having another format is to be discarded. 

[0084] Although the invention has been described with respect to a limited 
number of embodiments, those skilled in the art will appreciate that 



-15- 



other embodiments may be constructed within the spirit and scope of 
the following claims. 
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